feat: per-host caching#1974
Merged
thomas-zahner merged 6 commits intolycheeverse:masterfrom Dec 30, 2025
Merged
Conversation
This caching mechanism is independent of the lychee cache file.
Cache misses must only be tracked internally. Previously cache misses were tracked twice.
mre
approved these changes
Dec 29, 2025
Member
mre
left a comment
mre
left a comment
There was a problem hiding this comment.
Sorry about the mess. That means that we kept checking repeated URLs to the same hosts. On the plus side, with this being merged, we should see another performance bump for websites with many identical links. Only added a few very minor syntactic suggestions. Semantically, it's all good, so feel free to ignore. Great work!
Co-authored-by: Matthias Endler <[email protected]>
thomas-zahner
force-pushed
the
per-host-caching
branch
from
1d38d6b to
5d2cd31
Compare
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This actually implements per-host caching.
Previously there were some structures and methods which made it look like it was implemented. But upon closer inspection I realised this was not the case.
In the process I've also tried to clarify the difference between persistent inter-process caching (lychee cache file) and this process internal per-host caching in some places. Note that the requests which are cached in this "new" way do not appear as "cached" in the progress output. (from lychee cache
[200] http://localhost:9000/ | OK (cached)vs per-host cached[200] http://localhost:9000/) I've considered doing that as well, but that would probably require quite an additional effort without really much UX improvement. The per-host statistics cache rate is accurate and does count the per-host cache hits as such.Closes #1815 and #1593